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Executive Summary 


In recent years, interest has increased in preschool programs that promote low-income chil- 
dren’s early learning and development. Although research in this area has focused mostly on 4- 
year-olds, a growing number of 3-year-olds attend publicly funded preschool. In fact, in Head 
Start — a federally funded early childhood education program — the percentage of 3-year-olds 
rose from 24 percent in 1980 to 40 percent in 2013. 1 

Increased attention has also been focused on identifying promising strategies to enhance 
young children’s social, emotional, and behavioral development. A number of mostly small- 
scale studies demonstrate that classroom-based approaches can improve these outcomes for 4- 
year-old children. 2 However, a notable gap in the evidence base is whether the benefits of these 
social-emotional strategies can extend to younger children in the classroom. 

This report presents exploratory impact findings for 3 -year-old children from the Head 
Start CARES (Classroom-based Approaches and Resources for Emotion and Social skill pro- 
motion) demonstration. The Head Start CARES demonstration was a large-scale randomized 
controlled trial implemented in Head Start centers across the country. The demonstration tested 
the effects of three classroom-based approaches that each had a distinct theory, developed by 
the Head Start CARES research team, 3 of how to improve children’s social-emotional compe- 
tencies. Called “enhancements” because they were intended to complement and to enrich exist- 
ing Head Start classroom practices, the three approaches are The Incredible Years Teacher 
Training Program, or “The Incredible Years”; 4 Preschool PATHS (Promoting Alternative 
Thinking Strategies), or “PATHS”; and a one-year version of Tools of the Mind focused on 
play, or “Tools of the Mind— Play.” 5 The demonstration was conceived and sponsored by the 
Office of Head Start and the Office of Planning, Research and Evaluation in the Administration 
for Children and Families, U.S. Department of Health and Human Services. MDRC, a nonprof- 


'larullo, Aikens, Moiduddin, and West (2010); Child Trends (2014). 

2 Reid, Webster-Stratton, and Hammond (2003); Domitrovich, Cortes, and Greenberg (2007); Barnett et al. 
(2008). 

3 The Head Start CARES team developed the theory of change for each approach based on the training and 
curricular materials and research papers from each one. 

4 The Teacher Training Program is one of three Incredible Years programs; the other two are the child- 
focused Dinosaur School and the parent-focused Parent Program. 

5 Tools of the Mind— Play, a one-year program that promotes children’s learning through structured 
“make-believe” play, is adapted from the original two-year “Tools of the Mind” program. In Tools of the 
Mind— Play, teachers were trained for only one year in the model (instead of two years, as is typical in the 
Tools of the Mind program) and it was implemented as an enhancement to the existing curricula in the Head 
Start CARES program sites. 
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it, nonpartisan education and social policy research organization, conducted the demonstration 
in collaboration with MEF Associates and several academic partners. 


Head Start CARES: Earlier Findings and a Preview of 
Findings for 3-Year-Olds 

Two prior reports on Head Start CARES looked at all classrooms in the demonstration, which 
included classrooms with 4-year-olds only and “mixed-age” classrooms with both 3-year-olds 
and 4-year-olds. This work confinned that, on average, the enhancements were satisfactorily 
implemented in the full set of classrooms in the study, leading the research team to conclude 
that Head Start CARES provided a fair test of large-scale implementation of the three enhance- 
ments. 6 Furthermore, impact findings for the full sample of Head Start C ARES classrooms indi- 
cate that two of the three enhancements — The Incredible Years and PATHS — improved 4- 
year-old children’s social-emotional outcomes, with impacts of small to moderate size. 7 

The findings presented in this report suggest that the benefits of social-emotional en- 
hancements can extend to 3 -year-olds in mixed-age Head Start classes that include both 3- and 
4-year-old children. When considered as a group, the enhancements produced improvements in 
teachers’ reports of 3-year-olds’ social behaviors and closeness with their teachers. These over- 
all impacts appear to have been driven primarily by The Incredible Years, which is consistent 
with the hypothesis that The Incredible Years may be more accessible to 3 -year-olds than are 
PATHS and Tools of the Mind— Play. 

However, as discussed in more detail below, these questions were not the main focus of 
the study, and the conclusions that can be drawn from this analysis are limited because of the 
sample sizes, data sources, and measures available for the analysis. In addition, the pattern of 
impacts on 3 -year-olds’ social-emotional outcomes does not clearly align with the impacts on 
teacher practice and classroom climate in the classrooms serving these children. This finding 
raises questions about the findings that warrant further exploration, including what additional 
mechanisms might account for the impacts of the enhancements on 3 -year-olds’ social and 
emotional competencies. 


6 Mattera, Lloyd, Fishman, and Bangser (2013). 

Morris et al. (2014). While the full sample described in Morris et al. (2014) included all the Head Start 
CARES classrooms, it focused only on the 4-year-olds in those classrooms when discussing the analysis of 
child outcomes. 
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The Head Start CARES Demonstration 

Head Start CARES used a rigorous random assignment research design in which Head Start 
centers were randomly assigned to receive one of the three social-emotional enhancements or to 
a control group in which the curricula used in the Head Start program were carried out as usual 
without any of the enhancements. The enhancements were implemented between the fall of 
2010 and the spring of 201 1. 

The Head Start CARES enhancements are described in more detail in Box ES.l. They 
were designed for 4-year-olds, and, accordingly, the theory of change underlying these en- 
hancements is most relevant for this age group. 8 As shown in Figure ES.l, the enhancements 
share a common goal of improving children’s social, emotional, and behavioral competencies. 
The overarching hypothesis is that the enhancements influence children through changes in 
teacher practice and, in turn, classroom climate. However, each enhancement is thought to have 
a different mediating pathway by which it improves young children’s social-emotional out- 
comes. Enhancement-driven improvements in children’s social, emotional, and behavioral 
competencies might, in turn, lead to improvements in children’s cognitive and pre-academic 
skills. 9 However, the enhancements did not directly target those skills. 


The Current Analysis: An Opportunity to Explore the Impacts of 
the Head Start CARES Enhancements on 3-Year-Olds 

Head Start CARES provides a unique opportunity to explore the impacts of social-emotional 
enhancements on 3-year-olds in mixed-age classrooms. Yet, there are limitations to this analysis 
that have implications for the conclusions that can be drawn: 

• Lack of clear theory and evidence base. At the outset of the study, it was 
not clear whether and how 3 -year-olds might benefit from the enhancements. 

Much of the prior theory and intervention research in this area have focused 
primarily on 4-year-olds, with few insights into how 3 -year-olds could be af- 
fected. Given this uncertainty, Head Start CARES was not explicitly de- 
signed to test the impacts of the enhancements on 3-year-olds. As discussed 
below, the design of the impact analysis — including the sample sizes and 
data sources that were available — has several implications for what can be 
learned from it. 


8 As noted earlier, the theory of change was developed by the Head Start CARES evaluation team. 

9 Pre-academic skills are the early language, literacy, and math skills that underlie learning in elementary 
school. For example, in preschool, children learn to identify letters and the sounds that letters make, to provide 
a foundation for reading in kindergarten. 
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Box ES.1 


The Head Start CARES Enhancements 


• The Incredible Years Teacher Training Program (The Incredible Years) is designed to 
enhance children’s social-emotional development. Teachers are trained to create an or- 
ganized classroom that supports children’s ability to leam by watching others (“social 
learning”) and children’s behavior regulation in the context of positive teacher-child re- 
lationships. In an Incredible Years classroom, the teacher uses praise, clear commands, 
and consistent limit-setting to encourage appropriate and positive behaviors instead of 
singling out children who are misbehaving. For example, during an activity where chil- 
dren are asked to sit quietly, the teacher might say, “1 really like the way Juan is sitting 
with his hands in his lap.” 

• Preschool PATHS (PATHS) is an instructional approach to enhancing children’s social- 
emotional development through lessons and activities focused on children’s understand- 
ing of emotions (“emotion knowledge”) and social problem-solving skills, as well as 
through teacher modeling and support. In a PATHS classroom, teachers talk about their 
feelings and encourage children to think about their and others’ feelings in order to help 
children understand and leam about emotions in the context of social interactions. For 
example, in a group activity, the teacher might point out facial cues, like a smile, that 
show that children are feeling happy. 

• Tools of the Mind— Play requires teachers to restructure the room and school day, with 
large blocks of time devoted to supporting and structuring (scaffolding ) children’s 
make-believe play and role-playing games. By scaffolding students’ play, teachers aim 
to enhance the children’s planning skills, understanding of social roles, memory and ca- 
pacity for focused attention, and social-emotional understanding. For the Head Start 
CARES study, Tools of the Mind developers compressed the original two-year curricu- 
lum into a one-year enhancement focused on play, the central element of Tools of the 
Mind. For example, in a Tools of the Mind— Play classroom, a child might draw a pic- 
ture showing that she intends to play house and will be the mother. Through a series of 
exchanges with the child, the teacher would seek to expand the complexity of the child’s 
role play by asking questions like, “What will you do as the mother? How could you 
make dinner for your child? How would you get the food to cook dinner?” In doing so, 
the teacher helps to build the child’s self-regulation skills, planning skills, and ability to 
assume various perspectives through the role-playing activity. 


* 

Scaffolding is the act of helping a child accomplish a challenging task or acquire a skill that is just be- 
yond the child’s current ability level. 
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• Limited statistical power. All of the classes in Head Start CARES served 4- 
year-olds, but only a subset served both 3- and 4-year-old children. The im- 
pact analysis therefore has somewhat limited power to detect statistically sig- 
nificant impacts on mixed-age classrooms and the 3-year-olds in those class- 
rooms, especially impacts that are small or moderate in magnitude. 10 This is 
particularly relevant for separate tests of the impacts of the individual en- 
hancements on teachers’ practices and other class-level outcomes. 

• Small set of child outcomes. Data were not collected on 3 -year-old chil- 
dren’s executive function skills, 11 understanding of emotions (emotion 
knowledge), or social problem-solving skills — key outcomes that are the 
foci of the PATHS and Tools of the Mind— Play enhancements. 12 This yields 
uneven infonnation about the potential effectiveness of the enhancements for 
3-year-olds. 

• Reliance on teacher reports. Last, all the social-emotional outcomes ex- 
plored for 3-year-olds in this analysis were measured by teacher reports. 
Teacher reports are infonnative because the way in which teachers see the 
children in their classes and build relationships with them can shape chil- 
dren’s early schooling experiences. However, exclusive reliance on teacher 
reports can be a limitation; teachers’ ratings may be influenced by their own 
perceptions, and teachers who were trained in the Head Start CARES en- 
hancements might perceive children’s behavior differently from those who 
did not receive this training, regardless of whether children’s actual behav- 
iors changed. 

In light of these limitations, the impact results presented here are viewed as an oppor- 
tunity to add to the literature and to generate hypotheses to guide future intervention research in 
this area. The findings are not yet intended to be used to inform policy or practice. 


^Statistically significant impacts are those that are unlikely to have occurred by chance alone. 

1 'Executive function skills consist of the ability to flexibly shift attention from one piece of information to 
another, the ability to control one’s immediate or automatic response in favor of a planned response (inhibitory 
control), and working (or short-term) memory. 

12 The Incredible Years also targeted executive function skills. However, because data were collected for 
two out of the three key outcomes targeted by The Incredible Years, the lack of information about executive 
function skills is thought to be less of an issue for establishing the effects of the enhancement on 3-year-olds. 
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Characteristics of Grantees, Centers, Classrooms, and 
Children in the Current Sample 

This impact analysis uses a subset of grantees, centers, classrooms, and children from the full 
sample in the Head Start CARES study. 13 The sample for the larger Head Start CARES study 
was designed to reflect the racial, ethnic, and cultural diversity of Head Start children across the 
country, but was not selected to be statistically representative of all Head Start grantees in the 
United States. The 3-year-old children included in this analysis were served in 155 mixed-age 
classrooms located in 56 Head Start centers within 9 of the 17 grantees in the full Head Start 
CARES sample. These nine grantees were located in five states spread across three regions of 
the country (Midwest/Plains, West, and South) and varied on such characteristics as size, racial 
or ethnic composition of the children served, and number of participating centers. 

A typical Head Start CARES mixed-age classroom had a minimum of one lead teacher 
and one assistant teacher. The Head Start CARES classrooms that served 3-year-olds were simi- 
lar to classrooms in a nationally representative sample of Head Start centers on levels of emo- 
tional support, but had somewhat higher levels of classroom organization and instructional sup- 
port, as measured by the Classroom Assessment Scoring System (CLASS), a widely used 
measure of classroom climate. 14 Head Start CARES teachers in classrooms that served 3-year- 
olds, however, looked similar to the general population of Head Start teachers on characteristics 
like educational attainment and years of teaching experience. 15 In the current sample of mixed- 
age classrooms, an average of three classrooms per center participated in the Head Start CARES 
demonstration. 


Impacts of Head Start CARES Enhancements on Teacher 
Practice, Classroom Climate, and 3-Year-Olds’ Social-Emotional 
and Pre-Academic Outcomes 

In the absence of strong a priori hypotheses about how each enhancement would affect 3-year- 
olds and because greater statistical power can be leveraged with pooled research questions (that 
is, the statistical significance of smaller effect sizes can be detected with more certainty when 
tested with a larger sample), the analysis first tested for impacts with the data combined across 

1 3 A grantee is the local public or private nonprofit agency that has been designated a Head Start agency. 

I4 ln the sample of mixed-age Head Start CARES classrooms, average CLASS scores were 5.32 for Emo- 
tional Support, 4.84 for Classroom Organization, and 2.70 for Instructional Support. By comparison, in the 
nationally representative Head Start Family and Child Experiences Survey (FACES) sample, which included 
classrooms that served 4-year-olds only and mixed-age classrooms that served both 3- and 4-year-olds, average 
CLASS scores were 5.30 for Emotional Support, 4.70 for Classroom Organization, and 2.30 for Instructional 
Support (Moiduddin et al., 2012). 

15 U.S. Department ofHealth and Human Services, Administration for Children and Families (2012). 
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all three enhancements rather than for each enhancement individually. This analysis was con- 
ducted to assess whether the delivery of any social-emotional enhancement in mixed-age class- 
rooms might improve 3-year-olds’ social and emotional competencies. The results indicate the 
following: 

• When tested as a group, the enhancements increased levels of teachers’ 
social-emotional instruction but did not affect other aspects of teacher 
practice or classroom climate in mixed-age classrooms. 

As shown in Table ES.l, teachers in enhancement classrooms showed statistically sig- 
nificantly higher levels of social-emotional instruction than their control group counterparts. 
However, these impacts did not translate into statistically significant impacts on other aspects of 
teacher practice, such as classroom management and scaffolding, or classroom climate. 

• When considered as a group, the enhancements improved teacher re- 
ports of 3-year-olds’ social behaviors and closeness with their teachers. 

However, as shown in Table ES.l, no statistically significant impacts were found on 
teacher reports of 3-year-old children’s behavior problems, interpersonal skills, learning behav- 
iors, or conflict with their teacher when data were pooled across the enhancements. 

To isolate the impacts of the individual enhancements, separate analyses tested the im- 
pacts of each enhancement on teacher practice, classroom climate, and teacher reports of 3- 
year-olds’ social-emotional outcomes. The results, shown in Table ES.2, indicate the following: 

• The Incredible Years did not produce the expected statistically signifi- 
cant impacts on teachers’ use of classroom and behavior management 
strategies in mixed-age classrooms. The Incredible Years did, however, 
improve teacher reports of 3-year-olds’ social behaviors and closeness 
with teachers. 

The absence of statistically significant impacts on teachers’ use of classroom and be- 
havior management strategies is surprising, since these are central foci of The Incredible Years 
training, and The Incredible Years significantly improved teachers’ classroom management 
practices in the full Head Start CARES sample. 16 Despite the absence of statistically significant 
impacts on Incredible Years teachers’ classroom management practices in mixed-age class- 
rooms, Incredible Years teachers still reported statistically significantly higher levels of 3-year- 


16 Morris et al. (2014). The analysis did not test for whether there are statistically significantly different im- 
pacts between the full (4-year-old and mixed-age) and mixed-age-only samples of classrooms. Moreover, be- 
cause the mixed-age classrooms differ from the full sample of classrooms on a number of characteristics, it is 
not clear what might be driving any differences in the impact estimates. 
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Table ES.l 

Summary of Impacts on Outcomes for Mixed-Age Classrooms and 
3-Year-Old Children, Enhancements Pooled 



Control 

Program 

Difference 




Group 

Group 

(Program vs. 

Standard 

Effect 

Outcome 3 

Mean 11 

Mean 

Control) 

Error 

Size" 

Observed teacher practice outcomes 11 






Classroom management (1-5) 

4.00 

3.94 

-0.06 

0.12 

-0.10 

Social-emotional instruction (1-5) 

1.84 

2.08 

0.24 * 

0.14 

0.39 

Scaffolding (1-5) 

1.49 

1.59 

0.11 

0.11 

0.23 

Observed classroom climate outcomes ' 1 






Emotional support (T-7) 

5.52 

5.41 

-0.12 

0.13 

-0.19 

Classroom organization (1-7) 

5.07 

4.91 

-0.16 

0.16 

-0.20 

Instructional support (T-7) 

2.45 

2.47 

0.03 

0.14 

0.03 

Literacy focus (1-7) 

1.49 

1.57 

0.08 

0.08 

0.17 

Teacher-reported child outcomes 






Social-emotional outcomes" 






Behavior problems (0-52) 

6.55 

6.20 

-0.35 

0.92 

-0.04 

Social behaviors: social skills (0-60) 

38.61 

41.50 

2.89 ** 

1.29 

0.27 

Social behaviors: interpersonal skills (1-7) 

5.34 

5.39 

0.06 

0.09 

0.06 

Learning behaviors (1-7) 

4.42 

4.58 

0.16 

0.11 

0.17 

Closeness with the teacher (1-5) 

4.14 

4.28 

0.15 ** 

0.07 

0.22 

Conflict with the teacher (1-5) 

1.81 

1.74 

-0.06 

0.09 

-0.08 

Pre-academic skills f 






General knowledge (1-5) 

2.43 

2.60 

0.17 

0.15 

0.18 

Language and literacy (1-5) 

2.15 

2.30 

0.15 

0.12 

0.15 

Mathematical thinking (T-5) 

2.14 

2.25 

0.11 

0.14 

0.13 

Sample size 8 






Centers 

14 

42 




Classrooms 

40 

115 




Children 

220 

713 





SOURCES: MDRC calculations based on the observational assessments and teachers’ reports. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Rounding may cause slight discrepancies in sums and differences. 
a The rating scale for each outcome measure is shown in parentheses, from low to high. 


(continued) 
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Table ES.l (continued) 


b The control group means reported in this table are covariate-adjusted and were estimated using models in 
which data for all three enhancements were pooled. Separate indicators for each of the enhancements were not 
included as covariates in the models. Some discrepancies in control group means may appear across tables due to 
differences in model estimation for the pooled and by-enhancement impacts. 

c Effect size is calculated by dividing the impact of the program (the difference between the means for the 
program group and the control group) by the standard deviation for the control group. 

d All observed teacher practice outcomes were measured using the Adapted Teaching Style Rating Scale (Raver 
et al., 2012). All observed classroom climate outcomes were measured using the Classroom Assessment Scoring 
System (Pianta, La Paro, and Hamre, 2008). 

e The behavior problems outcome was measured using the Behavior Problems Index (Zill, 1990); social 
behaviors: social skills were measured using the Social Skills Rating Scale (Gresham and Elliot, 1990); social 
behaviors: interpersonal skills were measured using the Interpersonal Skills subscale of the Cooper-Farran 
Behavioral Rating Scales (Cooperand Farran, 1991); learning behaviors were measured using the Work-Related 
Skills subscale of the Cooper-Farran Behavioral Rating Scales; and closeness with the teacher and conflict with 
the teacher were measured using the Student-Teacher Relationship Scale (Pianta, 2001). 

'Pre-academic skills were measured using the Academic Rating Scale (National Center for Education Statistics, 
n.d.) 

®For all variables in the table, data are available for at least 96 percent of the sample. 


olds’ social behaviors in mixed-age classrooms, including cooperation, assertion, and self- 
control (not shown in table), as well as closeness with their teachers. These impacts were mod- 
erate to large in magnitude. However, no statistically significant effects were found on class- 
room climate or teacher reports of 3 -year-olds’ behavior problems, interpersonal skills, learning 
behaviors, or conflict with their teachers. 

• As expected, PATHS and Tools of the Mind— Play improved teachers’ 
social-emotional instruction and scaffolding of children’s play, respec- 
tively, in mixed-age classrooms. But there is little evidence to suggest 
that either enhancement improved teacher reports of 3-year-olds’ social- 
emotional outcomes. 

As shown in Table ES.2, teachers in PATHS mixed-age classrooms engaged in statisti- 
cally significantly higher levels of social-emotional instruction compared with their control 
group counterparts. This included teaching children about emotions, supporting children’s ex- 
pression and regulation of emotions, and facilitating children’s social problem-solving and un- 
derstanding of their peers’ emotions (not shown in table) — all of which is consistent with 
PATHS training. Teachers in Tools of the Mind— Play classrooms also showed higher levels of 
overall scaffolding of children’s play. These impacts are consistent with the central focus of 
Tools of the Mind— Play. However, these improvements in teacher practice did not lead to 
changes in overall classroom climate for either enhancement, nor did they translate into a con- 
sistent pattern of impacts on 3-year-olds’ teacher-reported social-emotional outcomes. 
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Head Start CARES Demonstration 


Table ES.2 


Summary of Impacts on Outcomes for Mixed-Age Classrooms and 3-Year-Old Children, by Enhancement 




| The Incredible Years (IY) 



Preschool PATHS 


j Tools of the Mind 

- Play 


Outcome 11 

Control 

Group 

Mean b 

Program 

Group 

Mean 

Difference 
(IY vs. 
Control) 

SE 

Effect 

Size c 

Program Difference 
Group (PATHS vs. 
Mean Control) 

SE 

Effect 

Size c 

Program 

Group 

Mean 

Difference 
(Tools vs. 
Control) 

Effect 
SE Size" 

Observed teacher practice 
outcomes'* 

Classroom management 
(1-5) 

4.00 

4.11 

0.11 

0.15 

0.20 

3.89 

- 0.11 

0.15 

- 0.21 

3.82 

-0.18 

0.15 

-0.32 

Social-emotional 
instruction (1-5) 

1.85 

1.97 

0.12 

0.15 

0.20 

2.42 

0.58 

*** 0.16 

0.94 

1.85 

0.00 

0.16 

0.00 

Scaffolding (1-5) 

1.48 

1.43 

-0.06 

0.13 

- 0.12 

1.60 

0.11 

0.14 

0.24 

1.76 

0.28 * 

0.14 

0.59 

Observed classroom climate 
outcomes'* 

Emotional support (1-7) 5.53 

5.47 

-0.06 

0.16 

-0.09 

5.45 

-0.07 

0.16 

- 0.12 

5.30 

-0.23 

0.17 

-0.37 

Classroom organization 
(1-7) 

5.07 

5.08 

0.01 

0.19 

0.01 

4.88 

-0.19 

0.19 

-0.24 

4.76 

-0.30 

0.20 

-0.38 

Instructional support (1-7) 

2.45 

2.39 

-0.06 

0.17 

-0.08 

2.70 

0.25 

0.18 

0.31 

2.36 

-0.09 

0.18 

- 0.12 

Literacy focus (1-7) 

1.49 

1.51 

0.02 

0.10 

0.04 

1.55 

0.06 

0.10 

0.12 

1.66 

0.17 

0.10 

0.34 

Teacher-reDorted child 
outcomes 

Social-emotional outcomes 

Behavior problems (0-52) 

e 

6.54 

5.89 

-0.65 

1.12 

-0.08 

5.96 

-0.59 

1.13 

-0.07 

6.74 

0.19 

1.13 

0.02 

Social behaviors: social 
skills (0-60) 

38.63 

44.41 

5.78 

*** | 45 

0.55 

40.94 

2.31 

1.46 

0.22 

39.16 

0.53 

1.46 

0.05 

Social behaviors: 
interpersonal skills (1-7) 

5.34 

5.46 

0.12 

0.11 

0.13 

5.34 

0.00 

0.12 

0.00 

5.38 

0.04 

0.11 

0.05 
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Table ES.2 (continued) 




[ The Incredible Years (IY) ! 

| Preschool PATHS | 

[ Tools of the Mind 

- Play 



Control 

Program 

Difference 




Program 

Difference 



Program 

Difference 




Group 

Group 

(IY vs. 



Effect 

Group (PATHS vs. 


Effect 

Group 

(Tools vs. 

Effect 

Outcome 3 

Mean b 

Mean 

Control) 


SE 

Size b 

Mean 

Control) 

SE 

Size b 

Mean 

Control) 

SE 

Size 

Learning behaviors (1-7) 
Closeness with the 

4.42 

4.64 

0.22 


0.14 

0.23 

4.60 

0.18 

0.14 

0.19 

4.51 

0.09 

0.14 

0.09 

teacher (1-5) 

Conflict with the teacher 

4.14 

4.41 

0.27 

*** 

0.08 

0.40 

4.25 

0.11 

0.08 

0.16 

4.20 

0.06 

0.08 

0.09 

(1-5) 

1.81 

1.65 

-0.15 


0.11 

-0.19 

1.81 

0.00 

0.11 

0.00 

1.77 

-0.04 

0.11 

-0.05 

Pre-academic skills f 

General knowledge (1-5) 

2.44 

3.02 

0.58 

*** 

0.16 

0.62 

2.49 

0.06 

0.16 

0.06 

2.30 

-0.14 

0.16 

-0.15 

Language and literacy 

2.15 

2.47 

0.32 

** 

0.14 

0.33 

2.26 

0.11 

0.14 

0.11 

2.15 

0.00 

0.14 

0.00 

(1-5) 

Mathematical thinking 

2.14 

2.55 

0.40 

** 

0.15 

0.49 

2.18 

0.04 

0.15 

0.05 

2.01 

-0.14 

0.15 

-0.16 

(1-5) 















Sample size 8 















Centers 

14 

14 





14 




14 




Classrooms 

40 

41 





37 




37 




Children 

220 

246 





226 




241 





SOURCES: MDRC calculations based on the observational assessments and teachers’ reports. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

SE represents standard error. 

a The rating scale for each outcome measure is shown in parentheses, from low to high. 

b The control group means reported in this table are covariate-adjusted and were estimated using models in which data for all three enhancements were 
pooled and a treatment indicator for each enhancement was included. Some discrepancies in control group means may appear across tables due to differences 
in model estimation for the pooled and by-enhancement impacts. 

c Effect size is calculated by dividing the impact of the program (the difference between the means for the program group and the control group) by the 
standard deviation for the control group. 
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Table ES.2 (continued) 

d All observed teacher practice outcomes were measured using the Adapted Teaching Style Rating Scale 
(Raver et al., 2012). All observed classroom climate outcomes were measured using the Classroom Assessment 
Scoring System (Pianta, La Paro, and Hamre, 2008). 

e The behavior problems outcome was measured using the Behavior Problems Index (Zill, 1990); social 
behaviors: social skills were measured using the Social Skills Rating Scale (Gresham and Elliot, 1990); social 
behaviors: interpersonal skills were measured using the Interpersonal Skills subscale of the Cooper-Farran 
Behavioral Rating Scale (Cooper and Farran, 1991); learning behaviors were measured using the Work-Related 
Skills subscale of the Cooper-Farran Behavioral Rating Scale; and closeness with the teacher and conflict with 
the teacher were measured using the Student-Teacher Relationship Scale (Pianta, 2001). 

Tre-academic skills were measured using the Academic Rating Scale (National Center for Education 
Statistics, n.d.). 

sFor all variables in the table, data are available for at least 96 percent of the sample. 


Last, analyses explored whether enhancement-driven improvements in 3 -year-olds’ so- 
cial-emotional competencies led to improvements in their pre-academic skills, first for the set of 
enhancements as a group (Table ES.l) and then separately for each enhancement (Table ES.2). 

• When considered together, the enhancements did not affect teacher re- 
ports of 3-year-olds’ pre-academic skills. Although teachers reported 
statistically significantly stronger pre-academic skills for 3-year-olds in 
Incredible Years classrooms than for their counterparts in control 
group classrooms, these findings are somewhat uncertain. 

As shown in Table ES.2, The Incredible Years’ impact estimates were consistent across 
teacher reports of three pre-academic skill domains — general knowledge, language and litera- 
cy, and mathematical thinking. It is important to consider the results of the full Head Start 
CARES impact analysis for 4-year-olds when interpreting these impacts on 3 -year-olds because 
infomiation about 4-year-olds’ pre-academic skills was collected from both teacher reports and 
direct assessments. Incredible Years teachers also reported statistically significant improve- 
ments in 4-year-olds’ pre-academic skills, but no statistically significant impacts were found on 
direct assessments of these skills. 17 The lack of convergence in findings across these two data 
sources and the potential for bias in teacher reports suggest that the impacts on 3 -year-olds’ 
teacher-reported pre-academic skills should be interpreted with caution. 


Discussion 

In sum, the findings suggest that it is possible for the benefits of social-emotional preschool in- 
terventions to extend to 3-year-olds, even if the interventions are primarily designed for 4-year- 
old children. When considered together, the Head Start CARES enhancements produced posi- 


17 Morris et al. (2014). 
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tive, statistically significant impacts on teacher reports of 3-year-olds’ social behaviors and 
closeness with their teachers. The exploration of impacts by enhancement further suggests that 
these impacts were primarily driven by The Incredible Years, and to a lesser extent by the other 
enhancements. The findings for social-emotional outcomes are generally consistent with the 
pattern of impacts on teacher-reported outcomes that were identified for 4-year-olds in earlier 
analyses from Head Start CARES. Furthermore, the impact results for 3-year-olds are consistent 
with the hypothesis that the behavioral focus of the Incredible Years enhancement may have 
been more accessible to 3-year-olds than the other two enhancements, which are more cogni- 
tively demanding. However, the expected statistically significant improvements in teacher prac- 
tice were not found in Incredible Years classrooms, making the mechanisms that might account 
for these effects less clear. 

These findings raise a number of questions for the field. First, can future studies repli- 
cate the findings presented here with other data sources, such as independent assessments of 
children’s behaviors? And, do the findings hold up in other contexts and classroom configura- 
tions? It will be important to leam whether future studies can confinn the findings discussed in 
this report before making decisions about appropriate policy and practice. 

Second, assuming that the results found here are replicated in future trials, what mecha- 
nisms might account for the impacts of the enhancements, and The Incredible Years in particu- 
lar, on 3-year-olds? It may be, for example, that mixed-age classrooms enhance children’s so- 
cialization through peer learning and promote child development by bringing children together 
who have different skill levels and capabilities. These dynamics might provide a strong platform 
for promoting young children’s social and emotional development when coupled with a class- 
room-based intervention. 

Third, do the PATHS or Tools of the Mind— Play enhancements improve social and 
emotional competencies for 3-year-olds that were not measured here? Measures of emotion 
knowledge, social problem-solving, and executive function — key outcomes targeted by the 
PATHS and Tools of the Mind— Play enhancements — were not available in this analysis. In- 
deed, earlier findings from Head Start CARES demonstrated that PATHS had small to moder- 
ate impacts on direct assessments of 4-year-old children’s knowledge of emotions and social 
problem-solving skills — two of its primary hypothesized outcomes. Therefore, it will be im- 
portant to investigate impacts on these outcomes for 3-year-olds before drawing conclusions 
about the effectiveness of the programs for younger children. 

Despite some shortcomings in the analysis and measures, these results based on a rigor- 
ous random assigmnent research design point to new potential directions for future research and 
the development of preschool interventions to enhance young children’s social-emotional com- 
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petencies. Since a substantial number of 3-year-olds attend preschool programs, the findings 
suggest that social-emotional interventions may be a strategy that deserves further investigation. 
Moreover, finding that 3-year-olds can benefit from such curricula suggests that there may be 
opportunities to augment these benefits for children who remain in preschool at age 4. Thus, 
future efforts might seek to develop and test social-emotional preschool program enhancements 
that include a second year of intervention. 
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